An on-line course can require students to generate and submit programming assignments or projects as part of the course curriculum. Programming assignments can allow for more complex inputs than, for example, standard type quizzes and/or tests. For example, a programming assignment can include programming code (computer code or software) and datasets. In another example, a programming assignment can include software that produces a specific formatted output and results. Examples of programming assignments can include, but are not limited to, computer science projects, statistics projects that use computer programs and code for statistical computing and graphics to analyze data (e.g., statistics projects implemented using R), and finance or business projects that use spreadsheets or other computational programs for analyzing and processing data.
In some implementations, a student can enroll in an online course in an on demand basis allowing the student to select when they want to take the course and, in some cases, the duration of the course. In these cases, a student is not limited to a fixed time for the offering and duration of the on line course. On demand grading of programming assignments can be challenging, especially in an on demand course case, because the assignment grader needs to be available essentially on a twenty-four hour, seven day a week, 365 days a year basis. An on line course provider needs to have the tools available for the grading of the programming assignments available to a student in a real-time, on demand basis in order to provide real-time feedback (e.g., a grade) to the student. Dependent on the course content, for example, a student can run the programming assignment locally on the student's computer and then submit the output of the programming assignment (e.g., numeric values, numeric ranges, or numeric or text expressions) to be compared against instructor-provided expected output. In another example, a student can submit the programming assignment as a file to be evaluated (e.g., executed or run) by a custom instructor-provided grader.